Dataflow integration

Note: To create or modify Dataflow links requires having a user account which is granted the role 'Economics/Dataflow Links' (see Roles).

Projects in Economics can be created by linking to documents in a Dataflow Version hierarchy. A Dataflow document can be mapped to a project, and the hierarchical structure of documents is mapped to a hierarchy of folders, sub-folders, and projects in Economics.

Using the link type 'Pivot and Aggregrate' it is possible to re-group the Dataflow source documents into a different hierarchy structure in Economics. The standard Dataflow link can only use the same hierarchy structure as the Dataflow source. Temporary data sets are created to store transformed and aggregated Dataflow data. These are not accessible to users but they do consume storage in the tenant database. An automatic scheduled cleanup process runs weekly by default (every Sunday at 00:01; all temporary data which are older than 5 days are deleted). The cleanup job is controlled by the Dataflow application settings 'Aggregation Data Cleanup Schedule' and 'Remove Aggregation Data Older Than' which can be modified by the IPS Administrator. The cleanup job should be set to run more frequently if the temporary data are found to be causing storage issues for the tenant database.

The Dataflow link is dynamic, so that updates in the source Dataflow documents are mapped to updates in the Economics projects and folders. Updates will be also triggered when the link's configuration is modified. The link can be set to Auto (Automatic) or Manual: if Automatic, any changes in the Dataflow hierarchy will be automatically synchronized to Economics. If Manual, changes will be synchronized when requested by a user.

To set up a Dataflow link requires configuration work in both Economics and Dataflow:

• In Economics, the link is configured by specifying the Dataflow Version that will be used for the source data, the section of the the Version hierarchy that will be used, the document types (Entity Types) of the Dataflow documents that will be linked to projects, and the Dataflow Scenarios that will be used in documents to be linked, that is, in order for a document to be linked it must contain at least one Dataflow Scenario that is configured in the link. Optionally, a Revision Tag can be specified to make use of document revisions that are not 'Latest'.
• The document templates in Dataflow Configuration each contain Economics Configuration settings. This specifies how the Dataflow variables are mapped to Economics variables (see Economics configuration). A document will be linked only when at least one of its variables has a defined mapping.

In the Manage Inputs workspace, the Hierarchy view shows the linking status of folders. If the folder label is marked with a small Dataflow icon then a link exists, for example: